DOCQHEHT BESUHE 



ED 088 383 



HE 005 264 



AUTHOR 
TITLE 



PUB DATE 
MOTE 

EDES FEICE 
DESCBIPTOBS 



IDENTIFIEBS 



Hursh, Daniel E. ; And Others 

Proctor's Discussions Of Students* Quiz Ferfomance 

With Students In a Self-Paced (ESI) Undergraduate 

Course. 

[73] 

12p. 

MF-$0,75 HC-$1,50 

Educational Research; *Higher Education; 
♦Individualized Instruction; ♦Performance Factors; 
Student Evaluation; ♦Student Testing; Undergraduate 
Study 

♦Personalized System of Instruction (PSI) 



ABSTRACT 

A personalized system of instruction (PSI) vas used 
to teach an undergraduate course in the principles and procedures of 
behavior modification. In most courses utilizing PSI, proctors grade 
quizzes and discuss the incorrect ansvers with the students. This 
study investigated the effect these discussions had on the students' 
quiz performance. The results indicated that the discussicns produced 
only slightly higher scores on initial attempts at quizzes. However, 
students had to retake only 3% of their quizzes when they experienced 
discussion, whereas they had to retake ^8% of their quizzes when they 
had no iiscussions. In addition, students answered questions 
concerning the general application of the principles and procedures 
of behavior modification both before and after taking the course. 
Their answers were then compared with the answers of a group of 
students who did not take the course. This comparison was made in 
order to see if a PSI course would produce a more general verbal 
repertoire than that which the course specifically required. The PSI 
students* answers to the general questions improved relative tc their 
initial answers, whereas to other students* answers did not improve. 
(Author) 
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Abstract 

A personalized system of instruction (PSI) was used to teach an undergraduate 
course in the principles and procedures of behavior modif ication. In most courses 
utilizing PSI, proctors grade quizzes and discuss the incorrect answers with the 
students. This study investigated the effect these discussions had on the students* 
quiz performance. The results indicated that the discussions produced only slightly 
higher scores on initial attempts at quzzzes. Houever, students had to retake only 
3% of their quizzes when they experienced discussion, whereas, they had to retake 
18% of their quizzes when they had no discussions. In addition, students ansi^ered 
questions concerning the general application of the principles and procedures of 
behavior modification both before and after taking the course. Their answers 
were then compared with the ansv/ers of a group of students who did not take the 
course. This comparison was made in order to see if a PSI course xirould produce 
a more general verbal repertoire than that which the course specifically required. 
The PSI students' ansvrers to the general questions improved relative to their 
initial answers, whereas, the other students' answers did not improve. 
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Since the personalized system of instruction (PSI) format was first proposed 
and implemented by Keller (1968), many studies have been carried out to compare 
its effectiveness with the more traditional lecture-exam format (e^g. , Alba & 
Pennypacker, 1972; Keller, 1968; McMlchael & Corey, 1969; Sheppard & ^^acDermot, 
1970), The results of these studies have indicated that the PSI format (or slight 
variations of it) produced higher student performance on final exams and higher 
student ratings of the course than the same materials presented in a lecture-exam 
format* 

The PSI format typically consists of five or six components. These components 
include: allovjlng students to proceed through the course at their own pace; dividing 
the course materials into a nuidber of unitsj providing study questions for each of 
the units; requiring students to pass quizzes over each unit with almost perfect 
scores before they are allowed to proceed; having students who have already mastered 
the course materials serve as proctors who correct the students* quizzes, discuss 
the incorrect answers with the students, answer any questions the students have and 
tutor students on difficult materials; and having only a few optional lectures. To 
implement these components requires a large investment of time, materials and money. 
Because of this large investment It is Important to experimentally Investigate which 
of the above components are necessary for PSI*s effectiveness and which may not be 
essential. 

The following studies (among others) have experimentally investigated the 
effectiveness of the above components. Lloyd (1971) found that instructor pacing 
produced higher and mora conslstant student performance than did allowing students 
to pace themselves (given fixed, end-of-^semester or end-of-term deadlines) < Semb 
(1973)cfound that dividing the course materials into many small units produced 
consistantly higher student performance than did dividing the course materials 
into larger units. Weaver & Miller (1973) also found that having many units 
T'Tith frequent quizzing vjas superior to having larger units with infrequent testing. 
The effects of having study questions (from vjhich quiz questions were taken) was 
investigated by Semb, Hopkins, & Hursh (1973). Their results indicated that the 
study questions the students were previously exposed to were correctly answered 
more than were questions to which the students had not previously been exposed. 
Jphnston & O'Neil (1973) & Semb (1973) found that the criterion- set for passing 
qtjlzzes is directly related to the level achieved by students. That is, high, 
"mastery** criteria maintain high quiz performance and lower criteria maintain 
lower quiz performance. Having proctors who interact with students when correcting 
some or all of the course quizzes has been shown to be more effective tlian not 
having proctors (Parmer, Lachter, Blausteln, & Cole, 1972). Lloyd, Garllngton, 
Lowry, Burgess, Euler, & Knowlton (1972) found that optional lectures do not 
serve as relnforcers for students' performance in the course. Thus, the experimental 
evidence supports the effectiveness of the use of small units, study questions, 
frequent quizzes with high '^mastery" criterion for passing and proctors to correct 
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and discuss quiz answers with students. However, it appears that self-pacing 
(given semester or term deadlines) and optional lectures are components that do not 
function to effectively maintain higher student performance. 

Keller (1968) emphasized the importance of usinjr proctors by noting that it 
allows for frequent quizzing, immediate feedback, discussion vjith students and 
tutoring of students. Tlie data (Farmer, er al., 1972) appear to support Keller's 
(1968) emphasis. However, proctoring involves many behaviors and it is not clear 
which of these many behaviors are functionally related to students' performance. 
The present Study evaluated whether proctors* discussions with students about 
their incorrect answers facilitated their test performance and whether or not the 
students preferred to have such discussions. Further, this study evaluated whether 
the students Trere able to apply the specific information they were required to learn 
during the course to general questions concerning the subject matter. 

Method 

Sttjdents and proctors 

Thirty-four students at the University of Kansas who were enrolled in a 
PSI course entitled "The Principles and Procedures of Behavior Ilodlflcation" took 
part in the present study. Four other undergraduates who had completed the course 
materials served as proctors for the course. 

Settings and class meetings 

A university classroom served as the setting for the class meetings. Class 
met for one hour, three days a vreek throughout the semester. With the exception 
of the first and last t^jo days students were free to attend class for the purpose 
of studying and/or taking quizzes whenever they chose. 

Procedures and experimental design 

Students in the course were required to pass all quizzes over the course 
readings with at least nine out of ten correct in order to receive an ''A" grade 
for the course. On each quiz the students were allowed as many retakes as necessary 
to achieve the 90% correct criterion. If a student failed to complete all 17 
quizzes by the end of the semester he was given a withdrawal instead of an "A" 
grade. 

The 34 students were divided into two groups. With one group of students 



Insert Figure 1 about here 



the proctors discussed incorrect answers on the first five quizzes but not on 
the second five quizzes. With the other group of students, the proctors did not 
discuss incorrect ansx^ers on the first five cuizzes but did not on the second 
five quizzes. On the final seven quizzes all students chose, on each quiz, whether 
or not to have the proctor discuss the incorrect anst/ers with them. If a proctor 
was to discuss an incorrect answer to a quiz question with a student, he gave the 
student prompts, discussed relevant Information and/or asked leading questions until 
the student correctly answered the question or it became obvious that the student 
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could not answer the question. If the si:udent could correctly answer the question 
during the discussion he was given credit: for it. If the proctor was not to discuss 
incorrect ansx^ers vjith the student, the proctor merely told the student his score 
and pointed out which answers were incorrect. A graduate student or professor 
vzas al^.3ys available in the classroom to answer any questions the students had about 
the readings, study questions or their answers to quiz questions* The graduate 
student or professor, hox\Tever, did not change the grading done by the proctors. 
After finishing the course, students completed a course evaluation that asked 
them which components of the PSl package they found helpful add hot; they would 
rate the course in comparison to the other courses they had taken. 

Before beginninj> the course and then again after they passed their last 
auiz^ all students in the course ans\^ered general questions over the course materials. 
These questions asked the student to describe behavior modification programs (e.g., 
to toilet train a child), design studies (e.p;., to experimentally analyze the effects 
of a behavior modification precedure) and describe the circumstances that v/ould 
lead one to use a punishment procedure to modify behavior. Another group of students 
enrolled in a PS I course in Nutrition also answered the general questions. From this 
group 17 students were chosen such that the mean of their scores matched the mean 
of the experimental classes scores and their individual scores matched the scores 
of 17 of the 34 students in the experimental class. Each of these 17 control 
students then answered the general questions a second time after an Interval 
equivalent to the time it took the student they were matched with to complete 
the course. All students irere simply asked to complete the test both times. There 
were no contingencies for completeness or accuracy. 

Observation and recording 

Data was collected on both quiz scores and answers to the general questions. 
Periodically interobserver reliability was assessed. On an answer-by-answer basis 
mean intergrader reliability was 98% for the quizzes and 68% for the general questions 

Results 

All students on their first attempt on all quizzes scored a mean of 96% correct. 
For quizzes on which students discussed their incorrect ansi>rers with the proctor 
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they scored a mean of 98% correct on their first attempts. For quizzes on which 
students did not discuss their incorrect answers with the proctors, they scored 
a mean of 94% correct on their first attempts. Thus, the discussion of incorrect 
answers produced only slightly higher scores on first attempts than did r:> discussion. 
However, when given a choice between discussion and no discussion students chose to 
discuss their incorrect answers with a proctor on 237 of the 238 occasions to choose 
(7 choice quizzes x 34 students) . In addition, students had to retake 18% of their 
quizzes when they were not given a chance to discuss their incorrect answers, V7bereas, 
when given a chance to discuss and potentially change their incorrect answers students 
had to retake only 3% of their quizzes. However, if students' initially Incorrect 
answers had not been corrected during the discussions they would have had to retake 
35% of their quizzes. 
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In the experimental course, students^ answers to the general questions 
contained a mean of 7.9 of the possible 26 points before they experienced the 
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course and a mean of 11.5 after they had completed the course. The students 
not in the experimental course scored a mean of 7.8 initially and 7*4 on their 
second attempt. Thus, students in the experimental course increased their 
performance on the general question^, xrhereas, the other students did not. 

A supplementary finding was that all students took and passed quizzes more 
frequently as they progressed throu^^h the course. Thus, a cumulative record 
of their quiz passing performance resembles the scallop-like pattern of responding 
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characteristic of animal (Ferster & Skinner, 1957) and human (Jfewhinney, Bostow^ 
Laws, Blumenfeld, & Hopkins, 1971) responding when under a fixed-interval 
schedule of reinforcement. 

On the course evaluation the students rated the experimental course as more 
Interesting than other courses they had taken (5.9 on a seven-point scale where 
1» much less interesting than and 7== much more interesting than) and more work 
than other courses (4.9 on a seven point scale where 1» much less than and 7== 
much more than) . On a seven-point scale (where 1= a hindrance and 7« very helpful) , 
students rated the helpfulness of having proctors at 6,4, being able to pace 
themselves at 6.9, having study guide questions at 7.0 and being required to pass 
quizzes vjithout least 9 out of id correct at 6.5. 

Discussion 

Discussions produced almost no increase in students* mean accuracy when 
initially answering quiz questions. However, it appears that students were better 
prepared to initially provide correct answers to the quiz questions ijhen they were 
not given a chance to discuss and potentially change their Incorrect answers. 
They had to retake 18% of their quizzes when they had no discussion but would 
have had to retake 35% of their quizzes if they had not been able to change some 
of their initially incorrect answers during discussions with the proctors. Giving 
the students the chance to change their initially incorrect answers in discussions 
with their proctors almost eliminated the need for retakes (only 3%). Another 
important finding vras that, after having experienced both discussions and no 
discussions, students almost always chose to have discussions when given the option. 

One potential criticism of PSI is that it appears to teach only a specifically 
defined and limited repertoire rather than setting the occasion for" a more general 
repertoire with respect to the subject matter. The general questions in this study 
vrere used to test whether this PSI format could produce a more general repertoire 
than that which was specifically required as part of the course itself. The results 
from these general questions appear to indicate that PSI can teach a more general 
repertoire. Having students from another course answer the same questions only 
provided a control for the possibility that the passage of time itself could 
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account for the Increased scores of students in the experimental PSI course. 
Further evaluations of this issue will need to include comparisons betx/een PSI 
techninues and other teaching methods. 

The results of the course evaluations indicated, as did earlier studies, that 
students report finding PSI more Interesting, even though it requires more 
work, than other courses they had taken. In the present study students were 
also asked hov7 helpful the different PSI components were. Their responses to 
these questions indicated that the students found each of the components very 
helpful. 
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